{
 "metadata": {
  "name": "",
  "signature": "sha256:b664ac2ba452bc821ed4bfb4cb72116ee81b6c75ed2a481b7f029ba2cb13d901"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A numerical agnostic pyrex class\n",
      "======================================================================\n",
      "\n",
      "TAGS: Outdated\n",
      "\n",
      "**NOTE**: This entry was last updated 2006-11-04 and contains information\n",
      "that is not relevant today (as of 2013).\n",
      "\n",
      "Overview\n",
      "--------\n",
      "\n",
      "Here is presented *!NumInd* (Numerical Independent Extension), an\n",
      "example of a class written in\n",
      "[Pyrex](http://nz.cosc.canterbury.ac.nz/~greg/python/Pyrex/). This class\n",
      "can wrap information from any object coming from Numeric, numarray or\n",
      "!NumPy without any dependency of those packages for compiling the\n",
      "extension. It lets you to create a uniform interface in both Python and\n",
      "C spaces. In addition, you can personalize it by adding new methods or\n",
      "properties.\n",
      "\n",
      "For this extension to work, you need a numerical package that supports\n",
      "the [array interface](http://numpy.scipy.org/array_interface.shtml). Any\n",
      "of these versions would be good enough:\n",
      "\n",
      "*\u00a0NumPy\u00a0(all\u00a0versions)\n",
      "*\u00a0Numeric\u00a0(>=24.2)\n",
      "*\u00a0numarray\u00a0(>=1.5.1)\n",
      "\n",
      "NumInd: a Numerical Independent Pyrex-based extension\n",
      "-----------------------------------------------------\n",
      "\n",
      "The !NumInd class shown below takes a Numeric/numarray/!NumPy object and\n",
      "creates another object that can be accessed in an uniform way from both\n",
      "Python and Pyrex (and hence, C) space. Moreover, it exposes an array\n",
      "interface so that you can re-wrap this object with any\n",
      "Numeric/numarray/!NumPy. All of these features are achieved without\n",
      "actually copying the data itself. This opens the door to the possibility\n",
      "to develop applications that supports the Numeric/numarray/!NumPy triad\n",
      "without a need to compile against any of them.\n",
      "\n",
      "**Warning**: This class supports mainly homogeneous datasets, but it\n",
      "wouldn't be difficult to support recarrays as well. This is a work\n",
      "in-progress anyway."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# This Pyrex extension class can take a numpy/numarray/Numeric object\n",
      "# as a parameter and wrap it so that its information can be accessed\n",
      "# in a standard way, both in Python space and C space.\n",
      "#\n",
      "# Heavily based on an idea of Andrew Straw. See\n",
      "# http://www.scipy.org/Cookbook/ArrayStruct_and_Pyrex\n",
      "# Very inspiring! :-)\n",
      "#\n",
      "# First version: 2006-03-25\n",
      "# Last update: 2006-03-25\n",
      "# Author: Francesc Altet \n",
      "\n",
      "import sys\n",
      "\n",
      "cdef extern from \"Python.h\":\n",
      "    ctypedef int Py_intptr_t\n",
      "    long PyInt_AsLong(object)\n",
      "    void Py_INCREF(object)\n",
      "    void Py_DECREF(object)\n",
      "    object PyCObject_FromVoidPtrAndDesc(void* cobj, void* desc,\n",
      "                                        void (*destr)(void *, void *))\n",
      "\n",
      "cdef extern from \"stdlib.h\":\n",
      "    ctypedef long size_t\n",
      "    ctypedef long intptr_t\n",
      "    void *malloc(size_t size)\n",
      "    void free(void* ptr)\n",
      "\n",
      "# for PyArrayInterface:\n",
      "CONTIGUOUS=0x01\n",
      "FORTRAN=0x02\n",
      "ALIGNED=0x100\n",
      "NOTSWAPPED=0x200\n",
      "WRITEABLE=0x400\n",
      "\n",
      "# byteorder dictionary\n",
      "byteorder = {'<':'little', '>':'big'}\n",
      "\n",
      "ctypedef struct PyArrayInterface:\n",
      "    int version          # contains the integer 2 as a sanity check\n",
      "    int nd               # number of dimensions\n",
      "    char typekind        # kind in array --- character code of typestr\n",
      "    int itemsize         # size of each element\n",
      "    int flags            # flags indicating how the data should be interpreted\n",
      "    Py_intptr_t *shape   # A length-nd array of shape information\n",
      "    Py_intptr_t *strides # A length-nd array of stride information\n",
      "    void *data           # A pointer to the first element of the array\n",
      "\n",
      "cdef void free_array_interface(void *ptr, void *arr):\n",
      "    arrpy = <object>arr\n",
      "    Py_DECREF(arrpy)\n",
      "\n",
      "\n",
      "cdef class NumInd:\n",
      "    cdef void *data\n",
      "    cdef int _nd\n",
      "    cdef Py_intptr_t *_shape, *_strides\n",
      "    cdef PyArrayInterface *inter\n",
      "    cdef object _t_shape, _t_strides, _undarray\n",
      "\n",
      "    def __init__(self, object undarray):\n",
      "        cdef int i, stride\n",
      "        cdef object array_shape, array_strides\n",
      "\n",
      "        # Keep a reference to the underlying object\n",
      "        self._undarray = undarray\n",
      "        # Get the shape and strides C arrays\n",
      "        array_shape = undarray.__array_shape__\n",
      "        self._t_shape = array_shape\n",
      "        # The number of dimensions\n",
      "        self._nd = len(array_shape)\n",
      "        # The shape\n",
      "        self._shape = <Py_intptr_t *>malloc(self._nd*sizeof(Py_intptr_t))\n",
      "        for i from 0 <= i < self._nd:\n",
      "            self._shape[i] = self._t_shape[i]\n",
      "        # The strides (compute them if needed)\n",
      "        array_strides = undarray.__array_strides__\n",
      "        self._t_strides = array_strides\n",
      "        self._strides = <Py_intptr_t *>malloc(self._nd*sizeof(Py_intptr_t))\n",
      "        if array_strides:\n",
      "            for i from 0 <= i < self._nd:\n",
      "                self._strides[i] = array_strides[i]\n",
      "        else:\n",
      "            # strides is None. Compute them explicitely.\n",
      "            self._t_strides = [0] * self._nd\n",
      "            stride = int(self.typestr[2:])\n",
      "            for i from self._nd > i >= 0:\n",
      "                self._strides[i] = stride\n",
      "                self._t_strides[i] = stride\n",
      "                stride = stride * array_shape[i]\n",
      "            self._t_strides = tuple(self._t_strides)\n",
      "        # Populate the C array interface\n",
      "        self.inter = self._get_array_interface()\n",
      "\n",
      "    # Properties. This are visible from Python space.\n",
      "    # Add as many as you want.\n",
      "\n",
      "    property undarray:  # Returns the underlying array\n",
      "        def __get__(self):\n",
      "            return self._undarray\n",
      "\n",
      "    property shape:\n",
      "        def __get__(self):\n",
      "            return self._t_shape\n",
      "\n",
      "    property strides:\n",
      "        def __get__(self):\n",
      "            return self._t_strides\n",
      "\n",
      "    property typestr:\n",
      "        def __get__(self):\n",
      "            return self._undarray.__array_typestr__\n",
      "\n",
      "    property readonly:\n",
      "        def __get__(self):\n",
      "            return self._undarray.__array_data__[1]\n",
      " \n",
      "    property __array_struct__:\n",
      "        \"Allows other numerical packages to obtain a new object.\"\n",
      "        def __get__(self):\n",
      "            if hasattr(self._undarray, \"__array_struct__\"):\n",
      "                return self._undarray.__array_struct__\n",
      "            else:\n",
      "                # No an underlying array with __array_struct__\n",
      "                # Deliver an equivalent PyCObject.\n",
      "                Py_INCREF(self)\n",
      "                return PyCObject_FromVoidPtrAndDesc(<void*>self.inter,\n",
      "                                                    <void*>self,\n",
      "                                                    free_array_interface)\n",
      "\n",
      "    cdef PyArrayInterface *_get_array_interface(self):\n",
      "        \"Populates the array interface\"\n",
      "        cdef PyArrayInterface *inter\n",
      "        cdef object undarray, data_address, typestr\n",
      "\n",
      "        undarray = self._undarray\n",
      "        typestr = self.typestr\n",
      "        inter = <PyArrayInterface *>malloc(sizeof(PyArrayInterface))\n",
      "        if inter is NULL:\n",
      "            raise MemoryError()\n",
      "\n",
      "        inter.version = 2\n",
      "        inter.nd = self._nd\n",
      "        inter.typekind = ord(typestr[1])\n",
      "        inter.itemsize = int(typestr[2:])\n",
      "        inter.flags = 0  # initialize flags\n",
      "        if typestr[0] == '|':\n",
      "            inter.flags = inter.flags | NOTSWAPPED\n",
      "        elif byteorder[typestr[0]] == sys.byteorder:\n",
      "            inter.flags = inter.flags | NOTSWAPPED\n",
      "        if not self.readonly:\n",
      "            inter.flags = inter.flags | WRITEABLE\n",
      "        # XXX how to determine the ALIGNED flag?\n",
      "        inter.strides = self._strides\n",
      "        inter.shape = self._shape\n",
      "        # Get the data address\n",
      "        data_address = int(undarray.__array_data__[0], 16)\n",
      "        inter.data = <void*>PyInt_AsLong(data_address)\n",
      "        return inter\n",
      "\n",
      "\n",
      "    # This is just an example on how to modify the data in C space\n",
      "    # (and at C speed! :-)\n",
      "    def modify(self):\n",
      "        \"Modify the values of the underlying array\"\n",
      "        cdef int *data, i\n",
      "\n",
      "        data = <int *>self.inter.data\n",
      "        # Modify just the first row\n",
      "        for i from 0 <= i < self.inter.shape[self.inter.nd-1]:\n",
      "            data[i] = data[i] + 1\n",
      "\n",
      "    def __dealloc__(self):\n",
      "        free(self._shape)\n",
      "        free(self._strides)\n",
      "        free(self.inter)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "An example of use\n",
      "-----------------\n",
      "\n",
      "In order to get an idea of what the above extension offers, try to run\n",
      "this script against the !NumInd extension:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import Numeric\n",
      "import numarray\n",
      "import numpy\n",
      "import numind\n",
      "\n",
      "# Create an arbitrary object for each package\n",
      "nu=Numeric.arange(12)\n",
      "nu.shape = (4,3)\n",
      "na=numarray.arange(12)\n",
      "na.shape = (4,3)\n",
      "np=numpy.arange(12)\n",
      "np.shape = (4,3)\n",
      "\n",
      "# Wrap the different objects with the NumInd class\n",
      "# and execute some actions on it\n",
      "for obj in [nu, na, np]:\n",
      "    ni = numind.NumInd(obj)\n",
      "    print \"original object type-->\", type(ni.undarray)\n",
      "    # Print some values\n",
      "    print \"typestr -->\", ni.typestr\n",
      "    print \"shape -->\", ni.shape\n",
      "    print \"strides -->\", ni.strides\n",
      "    npa = numpy.asarray(ni)\n",
      "    print \"object after a numpy re-wrapping -->\", npa\n",
      "    ni.modify()\n",
      "    print \"object after modification in C space -->\", npa"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can check the output here [test_output.txt](files/attachments/A_Numerical_Agnostic_Pyrex_Class/test_output.txt)\n",
      "\n",
      "See also\n",
      "--------\n",
      "\n",
      "* [\"Cookbook/Pyrex_and_NumPy\"]\n",
      "* [\"Cookbook/ArrayStruct_and_Pyrex\"] (The inspiring recipe)"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}